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(54) Eye tracking system 



(57) For providing a control signal responsive to the 'direction of look' of a person's eyes electrodes 1 and 
2 are placed on each side of the eyes and/or above and below the eyes to detect horizontal and/or vertical 
movement of the eyeballs by means of the unequal charge distribution which occurs naturally on the 
eyeballs 4 and 5. Such a control signal may be used by a patient incapacitated except for eye movement 
to effect communication by presenting the patient with a display of various functions which are selected by 
viewing the selected function. In military applications it could be used for target tracking. 




The drawing(s) originaDy filed was/were informal and the print here reproduced is taken from a later filed formal copy. 
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SPECIFICATION 
Eye-tracking system 

5 This invention relates to a method and apparatus to determine the 'direction of look' of a 5 
person's eyes, and provide a signal in response to the direction of look. 

According to one aspect of the invention a method of determining the orientation of an eyeball 
in its socket comprises the steps of mounting two electrodes in fixed positions on or closely 
adjacent to the head and deriving a potential difference between the electrodes responsive to 
10 charge distribution within the eyeball, the charge distribution being movable with the eyeball so 10 
that the potential difference is indicative of the orientation of the eyeball in its socket, and 
providing a signal responsive to the potential difference. 

The signal may be a control signal to be used, for example, by a severely disabled person, 
incapacitated except for eye movement. In such a case, communication or environmental control 
15 can be effected by the person viewing a display of various characters or functions which can be 15 
selected by moving the eyes to look at the appropriate character. A signal responsive to the 
direction of look could then be utilised for control or communication. 
The invention could also be used in military application for target tracking or other purposes. 
A potential difference can be detected around the eyes which changes with eye position. The 
20 cause of this potential is believed to be that when light enters the eye via the pupil and is 20 
focussed onto the retina, the retina becomes negatively charged and the front of the eye 
becomes positively charged. Thus a potential difference is generated between the front and rear 
of the eye. 

In the method according to the above invention the rate of change of potential difference may 
25 be measured to distiguish deliberate eye movements and first and second values of the potenial 25 
difference may be measured when the rate of change of the potential difference respectively 
exceeds and falls below a threshold value, the difference between the two values representing 
the extent of a deliberate eye movement. 
The method may further comprise the steps of calculating the average change of potential 
30 difference for eye movements between a pair of known orientations of the eyeball, recording the 30 
average change and thereby calibrating eyeball movement against change in potential difference. 

A further potential difference may be derived, indicative of the orientation of the head, and 
calibrated against head orientation by recording the potential difference generated at two or 
more known head orientations. 
35 The eyeball orientation may be combined with the head orientation to determine the 'direction 35 
of look' of the eyeball, and a control signal provided responsive to the 'direction of look'. 

The method may further comprise the steps of displaying symbols at known positions on a 
display so that the separation between any two symbols is known, comparing the change of 
'direction of look' on moving from a first to a second 'direction of look' with the known 
40 displacement between a first symbol corresponding to the first 'direction of look' and all other 40 
symbols, to determine a second symbol most nearly corresponding to the second 'direction of 
look'. 

According to another aspect of the invention apparatus for determining the orientation of an 
eyeball in its socket comprises two electrodes in fixed positions on or closely adjacent to the 
45 head, means to derive a potential difference between the electrodes responsive to charge 45 
distribution within the eyeball, the charge distribution being movable with the eyeball so that the 
potential difference is indicative of the orientation of the eyeball in its socket, and means to 
provide a signal responsive to the potential difference. The electrodes may be positioned one on 
each side of the head in operation, the potentials acquired by the two electrodes on lateral 
50 movement of the eyes relative to the head then being sum of the potentials. The apparatus may 50 
further comprise means to measure the rate of change of the potential difference, and threshold- 
ing means to distinguish deliberate eye movements. 

The apparatus may further comprise means to derive a further potential difference indicative of 
the head orientation, which means may comprise means to detect incidence of light from a 
55 scanning narrow light beam on a photocell or reflector, the photocell or reflector being adapted 55 
to be attached to the head. 

The apparatus may include comparison means for determining a drift error in an eyeball 
orientation signal corresponding to the potential difference. 

One embodiment of the invention will now be described, by way of example, with reference 
60 to the accompanying drawings of which: 60 

Figures 1A and 1B are diagrammatic plan views of a pair of eyeballs between electrodes; 

Figure 2 is a graph of potential difference against time showing the effect of sudden eye 
movement; 

Figure 3 is a block diagram of an electrode voltage measurement circuit; 
65 Figure 4 is a display of functions; 65 
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Figure 5 shows the steps required to print the word "How?" using the display of Fig. 4. 
and Figure 6 shows a circuit for estimating electrode voltage drift. 
Referring to Fig. 1A, two electrodes 1, 2 are positioned one against each side of the head (not 
shown) with the centre of each electrode aligned approximately with the centres of the eyeballs 
5 4 5 Conductive electrode jelly is used to lower skin resistance and ensure a good electrical 5 
contact. Light entering the eye via the pupil 7 is fbcussed onto the retina 9 causing the retina to 
become negatively charged and the front of the eye to become positively charged. With the 
electrodes positioned as described and the eye looking straight ahead, positive and negative 
charges are induced in approximately equal quantities on each electrode, and the potential 
10 difference between the electrodes would be expected to be close to zero. There is however, a 10 
small unbalanced, d.c. voltage due mainly to interaction between the skin and the electrodes. 

As'the eyes rotate, as in Fig. 1B, the charge distribution rotates with the eyeballs causing one 
of the electrodes, 1, to become net positively charged and the other, 2, to become net 
negatively charged. The potential difference E between the electrodes is believed to be propor- 
15 tional to the light magnitude, K and the angle of rotation from the 'straight-ahead' reference 15 
direction. 
Thus, 



20 



E=K0+C 

where C is the unbalanced d.c. level. 
From experiment. 



25 



35 



20 



Kfe15to30A*V 25 

The d c. voltage C depends upon many variable factors including the exact position of the 
electrodes, the contact pressure, the electrode jelly, the type of electrodes used and possibly 
changes within the skin. The electrodes are made with a large surface area (27X25 mm) to 
minimise some of these effects. C is eliminated by using electrode voltage differences. Any 
30 remainder, caused by C changing between voltage readings, can be corrected by the circuit to 30 
be described below. 
So, 



E A -E B =K(0 A -0J 35 

where A, B are two different eye orientations. Thus, the change in potential difference caused 
by eye movement is linearly proportional to the change is eyeball orientation. 

This linearity can be seen in Fig. 2 which shows a graph of electrode potential difference E 
against time. From t-0 to t=t A the voltage drifts randomly as C varies. At t=t A a deliberate 
40 eye movement begins. As the eyeball rotates from orientation A to orientation B, the electrode 40 
voltage changes in linear proportion to the angle of rotation. The difference, Ea-E A can thus be 
converted directly to an angle. 

The two electrodes positioned on the sides of the head detect lateral movements of the 
eyeball. Two further electrodes could be attached to the head one above and one below one 
45 eye to detect elevation movements. This provides a more useful system since the overall eye 45 
movement can be determined from the sum of the movements in the two places, and the user 
is not limited to viewing points on the same horizontal plane. 

Eye movement is detected by constantly measuring the rate of change of the potential 
difference between the electrodes and comparing it against a threshold rate of 2° per second 
50 When the eyes move deliberately from one position to another, they move at a rate greater than 50 
2°/sec. Thus the start and finish of eye movement is very easily detected from the very slow 
rate of change due to electrode drift. 

The circuit for employing the potential difference and measuring its rate of change is shown in 
Fig 3 The output from the electrodes is applied to very low noise differential amplifier 1 1. Low 
55 pass filter 13 gives a bandwidth of d.c. to 2 Hz to allow eye movement to be picked out from 55 
other, higher frequency, signals, present from, for example, brain and muscle actions. The 
filtered output is amplified by amplifier 15 with a gain of about 100, then passed through filter 
17, which is a notch filter at 50 Hz to eliminate 'mains' interference. A second low pass filter 
19 # eliminates any frequencies above 2 Hz introduced since the first one, 13, and further 
60 amplification is provided by amplifier 21 having a gain of 10.30. 60 
Output 27 from amplifier 21 is fed directly to a microcomputer 28. It also feeds a differentia- 
tor 23 which continuously measures the rate of change of the output signal, and comparator 25 
which compares the signal rate of change with the threshold value for eye movement of 2 /Sec. 
When the rate increases through this threshold a signal is provided on output 26 to enable the 
65 microcomputer to sample the signal value on output 27. When the rate fails through 2°/sec the 65 
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microcomputer again samples the signal value. 

The voltage level on output 27 is monitored by the computer for a predetermined time, 
typically 0.5 to 1 second. If the level remains approximately constant during this time, the 
computer 28 goes ahead with its calculation of eye movement. This time delay is imposed to 
5 distinguish deliberate looks from glances as the eyes move about naturally. It has been found 5 
that, unless the eyes are deliberately focussing on a particular point, the time spent at rest at 
any point is generally much less than 0.5 seconds. 

Until deliberate eye movement is detected at comparator 25, and a deliberate stare is regis- 
tered by the computer 28, the output 27 from amplifier 21 is also applied to a drift correction 

10 loop consisting of a comparator 29 and an integrator 31. Comparator 29 compares the signal 10 
level with a predetermined optimum level, chosen to suit the amplifier 21 and the microcomputer 
28. In this case the reference voltage applied to the comparator is +0.8V which is the mid- 
operating point of the analogue to digital converter in the BBC Micro. Output from the compara- 
tor 29 is applied to integrator 31 which acts to bring the comparator output to zero. The 

15 integrator output then rises and rails with signal level. If the comparator output is not zero, the 15 
integrator output increases or decreases with time, acting to restore the input difference to 
amplifier 21 to the optimum value. When eye movement is detected, the circuit is switched out 
at switch 30 controlled by the output of comparator 25. This prevents errors which would occur 
if the integrator 31 attempted to correct the large and rapid voltage changes due to eye 

20 movement. 20 
The difference between the samples output voltages at the start and finish of eye movement 
is calculated by the microcomputer. This difference represents a change in eyeball orientation. 
These measurements suffice to determine the 'direction of look' if it is assumed that the head 
does not move, but it is in fact impossible to keep the head perfectly still. For high precision, or 

25 circumstances in which the head is allowed to move freely it is necessary to measure head 25 
orientation in addition to eyeball orientation. One possible method of measuring head orientation 
is to use a very small photodetector attached to the head. A scanning light beam is directed 
generally towards the head and at some point during the scan it is detected at the head. The 
photodetector is made small so that it only detects light within a very narrow angle which 

30 enables the point of scan to be accurately determined. The area of scan and detector are 30 
arranged so that when the head is facing the scanner directly, the detector picks up the light at 
the centre of the scan. As the head turns, the point of detection of the beam moves away from 
the centre, and the precise head orientation can be determined from this. 
It is still of course necessary to keep the eyes horizontal, as this will not allow for head 

35 rotation about the horizontal axis from the back of the head to the front, although if necessary a 35 
second head orientation detector could be added to cover this rocking head movement. 
An alternative method of determining head orientation would be to use a gyroscope. 
Head orientation information, if it is obtained, is combined in the microcomputer with the 
eyeball orientation information, to give an overall indication of the 'direction of look' of a 

40 person's eyes. 40 
Before eye or head orientation can be determined, the detectors must be calibrated. For the 
eye calibration, the user is required to look repeatedly between certain pairs of fixed points. The 
average voltage change is recorded for each pair of points, thus providing a reference 'expected' 
voltage change for eye movements through any angle, because the voltage changes linearly with 

45 the angle. If two pairs of electrodes are used, each pair is calibrated separately. The calibration 45 
procedure could be conducted by an assistant, or by software or simply by the user according 
to a preset routine. The head detector can be calibrated in a similar way. 
Fig. 4 shows one possible function display 39 

Fig. 4 shows one possible function display 39 for use with the invention. Rows 4 to 6 carry 

50 the standard typewriter numeric characters; rows 1 and 3 carry sumbols for various control 50 
functions relating to the characters. For example, the PRINT function would activate a printer 
connected to the system for letter writing, say. PROG would put an associated microcomputer 
into programming mode, so that computer programs could be developed. Row 2 carries various 
environmental control symbols, such as lighting at two different points or intercom connection to 

55 two different rooms. There are of course very many possible functions that could be provided; 55 
Fig. 4 is intended purely as one example of a display. Furthermore, where the user has 
movement in one finger, for example, even more possibilities are opened up. The display is 
shown on a VDU, but it could of course be simply a fixed sheet of paper, or points about the 
room could be used as reference directions. It would also be possible to use the invention 

60 without any direction guides, by simply remembering that looking to bottom left activates one 60 
function, and to top right another, and so on. 

Referring to Fig. 4, each character or function is selected by the user viewing that symbol on 
the display. Before functions can be reliably selected however, calibration must be made as 
described above so that the expected voltage change is known for a change in direction of look 

65 between any two symbols, the co-ordinates of all symbols being stored in the computer. 65 
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Once calibration is complete, the user looks at a first symbol which has been designated as 
the starting reference point, and from there to the first desired function symbol. Fig. 5, as an 
example of the operation of the system, shows the steps required to print out the word 
"How?". The designated starting reference point for the screen of of Fig. 4 is the centre of the 

5 square showing REPEAT. This is chosen simply because it is fairly central to view, but the 5 
designated starting point could be at any one of the stored co-ordinates. 

The rate of change of electrode voltage is measured continuously by differentiator 23 of Fig. 
3, and as the eyes move (deliberately) from REPEAT towards SHIFT, the rate rises above the 
threshold of 2°/sec. T his prompts the microcomputer to sample and store t he voltage on output 

10 27. This is E^t. If the he ad orientation is being recorded, a reading is now made simulta ne- 10 
ouslv with the voltage sam ple. The high rate of change of voltage is maintained until the eye 
comes to rest on the next symbol, SHIFT, when the rate falls back below 2°/sec. 

The user must then stare at SHIFT for at least the predetermined minimum 'stare' time. If the 
voltage, which is now being measured continuously, remains approximately constant for this 

15 minimum time, the microcomputer takes the voltage reading, E^, and a head orientation 15 
reading if necessary. This "stare, time" condition is imposed to distinguish deliberate function 
selection from glances. The difference, Eg^-E^T, represents the angle through which the 
eyeballs have rotated. This is combined with the calculated change in head orientation, to give 
an overall change in 'direction of look'. 

20 The co-ordinates of the centre of each square are stored, so, from the calibration, the 20 
expected change in voltage between any two squares is known. The measured change is 
correlated against this information to determine which symbol is being viewed. The measured 
change is unlikely to be precisely the same as the expected change because of electrode drift 
during movement and also because the eyes might not focus exactly in the centre of a square, 

25 in which case the microcomputer selects the nearest symbol. The measured change is corrected 25 
by the computer to the perfect 'expected' change. The nature of the d.c. voltage, C is such that 
it tends to drift in the same direction for long periods of time. If the erro were allowed to 
accumulate, it would eventually lead the computer to a false conclusion in its estimate of the 
nearest function. 

30 So the measured voltage change is corrected to the value expected for a movement from 30 
REPEAT to SHIFT making allowance of course for any head movement. The control signal 
corresponding to SHIFT is then provided. In this case it is to activate the upper function of each 
square for the next selected function only (unless LOCK is that next function). The eyes then 
move to PRINT. The same process takes place — the computer samples the voltage and head 

35 orientation as the eyes leave SHIFT, and when they are at rest on PRINT, after the stare delay. 35 
The measured voltage change, Ep^-E^, is correlated against expected changes to find the 
closest, and this corrected value is used to update 6. The control program continues as shown 
in Fig. 5. Letters of the alphabet are addressed by a two-digit code. 
A further improvement can be made to the system by making some sophisticated allowance 

40 for electrode drift. Fig. 6 shows a circuit for estimating electrode voltage drift during non eye 40 
movement periods. 

The estimator is based on the theory of the steady state Kalman Filter. During non-eye 
movement periods the system can be adequately modelled by the following differential equations 



45 0 =d (1) 
d -n (2) 



55 0=6+v 



where v is a gaussian white noise process of spectral density and representing the measurement 



45 



where 

6 is the eye angle (voltage) 
50 d is the drift rate (voltage) 50 
n is the gaussian white noise process of spectral denity q. This noise represents any small 
changes in the drift rate during the estimation period. The eye angle 0 can be measured during 
the non-eye movement period and the measured angle 9 m is given by 



55 



noise. 

An estimator can be constructed using equations (1) and (2) by appending corrective terms 
60 proportional to the error between the predicted and measured eye angle. The estimator equa- 60 
tions are given by 

\ 8+ k, (0 m -&) 
d - k 2 (0 m -8) 
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where* denotes estimates. 
The steady state Kalman gains are given by 

5 k 2 = W 

The gains can be calculated off line and stored in look up tables. The transfer function from 
input & m to output v is: 

10 & k,s+k 2 



6 m s*+K lS +k 2 
and from 6 m to 3 is 

15 

3 k 2 s 



10 



15 



e m s*+k lS +k 2 



20 Typical values of q and r are: 20 

q = (0.001 ) 2 
r =(.05P 



25 Therefore oc 
k,=0.2 25 
K 2 =0.02 

The natural frequency of the drift estimator is Vk 2 =0.141 rad/sec. 
When the eye movement detector 43 indicates no eye movement the switch 44 is closed and 
30 the circuit acts to estimate 3 and 8 by integrating 30 




35 When an eye movement is detected, the switch 44 is opened and the first estimated angle 35 
voltage 0, is read by the computer on output 45. 

When the eye movement detector indicates that eye movement has ceased, the switch 44 is 
closed and the second estimated eye angle voltage 0 2 is read by the computer. This voltage is 
monitored by the computer for the predetermined minimum 'stare time' before the voltage 
40 change is calculated and responded to. 40 
The inputs 3(0) and 9(0) to integrators 46 and ^7 are constants of integration, added to the 
signal after it has been integrated in each case. 0(0) is reinitialised by 8 after each eye move- 
ment, so that the voltage & on output 45 is a cumulative total. 



45 CLAIMS 45 

1 . A method of determining the orientation of an eyeball in its socket comprising the steps 
of mounting two electrodes in fixed positions on or closely adjacent to the head and deriving a 
potential difference between said electrodes responsive to charge distribution within the eyeball, 
said charge distribution being movable with the eyeball so that said potential difference is 

50 indicative of the orientation of the eyeball in its socket, and providing a signal responsive to said 50 
potential difference. 

2. A method according to Claim 1 wherein the rate of change of said potential difference is 
measured to distinguish deliberate eye movements. 

3. A method according to Claim 2 and further comprising the step of measuring first and 

55 second values of said potential difference when the rate of change of said potential difference 55 
respectively exceeds and falls below a threshold value, the difference between said values 
representing the extent of a deliberate eye movement. 

4. A method according to any preceding claim and further comprising the steps of calculating 
the average change of potential difference for eye movements between a pair of known orienta- 

60 tions of the eyeball, recording said average change and thereby calibrating eyeball movement 60 
against change in potential difference. 

5. A method according to any preceding claim and further comprising the step of deriving a 
further potential difference indicative of the orientation of the head. 

6. A method according to Claim 5 and further comprising the step of calibrating said further 

65 potential difference against head orientation by recording the potential difference generated at 65 
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two or more known head orientations. , _ 

7. A method according to Claim 6 wherein said eyeball orientation is combined with said 
head orientation to determine the 'direction of look' of said eyeball. 

8. A method according to Claim 7 and further comprising the step of providing a control 
5 signal responsive to said 'direction of look'. ...... -j 

9. A method according to Claim 8 wherein said control signal is withheld until said direction 
of look* has been maintained for a predetermined time. 

10 A method according to Claim 7 and further comprising the steps of providing an initial 
reference direction, calculating the changes of orientation of the eyeball and of the head for each 

10 successive change in 'direction of look* and calculating said 'direction of look' from said initial iu 
reference direction and the sum of all of said changes of orientation. 

1 1 A method according to Claim 10, further comprising the steps of displaying symbols at 
known positions on a display so that the separation between any two symbols is known, 
comparing said change of 'direction of look' on moving from a first to a second 'direction of 

15 look' with the known displacement between a first symbol corresponding to said first 'direction 15 
of look' and all other symbols, to determine a second symbol, most nearly corresponding to 
said second 'direction of look'. 

12 A method according to Claim 11, further comprising the step of correcting the measured 
change of potential difference arising from said change of 'direction of look' to the value 

20 corresponding to the known displacement between said first and said second symbols. ZO 

13 A method according to any of Claims 7 to 12 wherein drift in potential difference 
between said electrodes is estimated during periods of no deliberate eye or head movement and 
the signals indicative of orientation corrected accordingly. 

14 A method according to any preceding claim wherein two pairs of electrodes are mounted 

25 on or closely adjacent to the head, the potential difference between the electrodes of one of 25 
said pairs being indicative of movement of the eyeball in its socket in a first plane and the 
potential difference between the electrodes of the other of said pairs being indicative of move- 
ment of the eyeball in its socket in a second, transverse plane. 
15. A method according to Claim 14 wherein said first plane is an elevation plane and said 

30 second plane is an azimuthal plane. 

16 Apparatus for determining the orientation of an eyeball in its socket compnsmg two 
electrodes in fixed positions on or closely adjacent to the head, means to derive a potential 
difference between said electrodes responsive to charge distribution within the eyeball, said 
charge distribution being movable with the eyeball so that said potential difference is indicative 

35 of the orientation of the eyeball in its socket, and means to provide a signal responsive to said 35 

potential difference. ... u 

17 Apparatus according to Claim 16 wherein said electrodes are positioned one on each 
side of the head in operation, the potentials acquired by the two electrodes on lateral movement 
of the eyes relative to the head then being the sum of the potentials. 

40 18 Apparatus as claimed in Claim 17, further comprising means to measure the rate of 40 
change of said potential difference, and thresholding means to distinguish deliberate eye move- 
ments. _ . . 

19 Apparatus as claimed in Claim 18, further comprising means to measure first and second 
values of said potential difference when the rate of change of said potential difference respec- 

45 tiveiy exceeds and falls below a threshold value, rates of change of said potential difference in 45 
excess of said threshold value indicating deliberate eye movement. 

20 Apparatus according to any of Claims 16 to 19 further comprising means for calculating 
the average change in potential difference for eye movements between a pair of known onenta- 
tions of the eyeball, and means for storing said average change. 

50 21. Apparatus according to any of Claims 16 to 20 further comprising means to derive a 50 

further potential difference indicative of the head orientation. 
22. Apparatus according to Claim 21 wherein said means to derive a further potential 

difference comprises means to detect incidence of light from a scanning narrow light beam on a 

photocell or reflector, said photocell or reflector being adapted to be attached to the head. 
55 23. Apparatus according to Claim 22, further comprising means to combine said eyeball 55 

orientation with said head orientation, and thereby to determine the 'direction of look' of the 

said eyeball. . 
24. Apparatus according to Claim 23, further comprising means to provide a control signal 

responsive to said direction of look. 

60 25. Apparatus according to any of Claims 16 to 24 including comparison means for deter- oU 
mining a drift error in an eyeball orientation signal corresponding to said potential difference, 
integrating means having a long time constant compared to the period of deliberate eye move- 
ment for integrating said drift error, the integrated error period being subtracted from the eyeball 
orientation signal to suppress said drift error. m 

65 26. Apparatus according to any of Claims 16 to 24, further comprising means for estimating o5 
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the drift in potential difference between said electrodes during periods of no deliberate eye 
movements and means to correct accordingly the signals indicative of orientation. 

27. Apparatus according to Claim 26 wherein said means for estimating said drift and said 
means to correct said signals comprise a Kalman Filter circuit. 
5 28. Apparatus according to any of Claims 16-27 including low pass filter means to which 5 
said eyeball orientation signal is applied to suppress variations arising from extraneous body 
sources. 

29. Apparatus according to Claim 28, including band stop fitter means to which said eyeball 
orientation signal is applied to suppress variations arising from supply system interference. 
10 30. Apparatus according to any of Claims 16 to 18, further comprising two electrodes 10 
positioned one above and one below the eye, the potential difference across said electrodes 
being indicative of the elevation of said eyeball. 

31. Apparatus substantially as hereinbefore described with reference to the accompanying 
drawings. 
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